Xml Export - Too long

Hi Everyone,

I created a script that open all modules of a project and for each module I generate an XML file "module_name.xml", then I export its content in this file (except OLE and pictures). Everything is working, but there is one thing : it takes really too much time...!

My question is simple : do you have any advice to make it faster ?

Thanks !!!!

Re: Xml Export - Too long
Mathias Mamsch - Mon Jun 26 10:39:47 EDT 2017

The answer depends largely on what your script does.

- What XML generation method do you use (DOORS internal perms, COM, Custom solution) ?

- What exactly means "too much time" ? Do you experience a slow down?

- Did you already do the obvious checks for memory management (i.e. make sure you leak no memory) ?

Another approach to make it faster is to parallelize. I made a parallelization framework once, in which I implemented for example an export of all modules content to SQLite. This way you can export multiple modules at the same time.

Regards, Mathias

Re: Xml Export - Too long
Joooris - Tue Jun 27 03:36:41 EDT 2017

Thanks for your answer Mathias.

- What XML generation method do you use (DOORS internal perms, COM, Custom solution) ?

This is a custom solution. I use the function to write in a file :

    Stream moduleXmlStream = write(exportedFileName)
    moduleXmlStream << "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
    [...]

Basically the layout of the XML is the following :

<Module name="moduleName">
    <Object id="001">
        <Attribute name="attributeName1" value="attributeValue1">
        <Attribute name="attributeName2" value="attributeValue2">
        <Attribute name="attributeName3" value="attributeValue3">
    </Object>
    <Object id="002">
        <Attribute name="attributeName1" value="attributeValue1">
        <Attribute name="attributeName2" value="attributeValue2">
        <Attribute name="attributeName3" value="attributeValue3">
    </Object>
    [...]
</Module>

- What exactly means "too much time" ? Do you experience a slow down?

I usually export 4 attributes for each object. I was testing the script on a test project with not so many modules/objects. Then I used it in a "real" project and I saw that it can takes 2 minutes (which is not really user friendly). So I was just wondering if there was 

- Did you already do the obvious checks for memory management (i.e. make sure you leak no memory) ?

I don't know how to handle this, except by deleting skip list I previously created.

Parallelization seems to be a good idea, but it also seems to be quit difficult to implement in DXL script. Can you give me more clue please ?

Thanks

Re: Xml Export - Too long
Mathias Mamsch - Tue Jun 27 05:27:59 EDT 2017

    Stream moduleXmlStream = write(exportedFileName)
    moduleXmlStream << "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
    [...]

<Module name="moduleName">
    <Object id="001">
        <Attribute name="attributeName1" value="attributeValue1">
        <Attribute name="attributeName2" value="attributeValue2">
        <Attribute name="attributeName3" value="attributeValue3">
    </Object>
    <Object id="002">
        <Attribute name="attributeName1" value="attributeValue1">
        <Attribute name="attributeName2" value="attributeValue2">
        <Attribute name="attributeName3" value="attributeValue3">
    </Object>
    [...]
</Module>

if this is a custom solution we need to look at your code, to see if we can make it any faster. Two minutes seems a lot of time for a small (0-2000 objects) module, especially if you do not export OLE objects, etc. 

The parallelization framework is already done, you can find it here (including a tutorial how it was made)

https://github.com/domoran/DXLParallels

Inside the parallels library you can also find the StreamEX class which allows you to abstract the output "device" similar to java. It contains methods for reading and writing to files without leaks, but would need to be migrated to be running ob 64bit. This can be helpful especially if you write a lot of files (to avoid the leaks that come with opening a stream).

Regards, Mathias

Re: Xml Export - Too long
Joooris - Fri Jun 30 03:40:33 EDT 2017

Hi,

Thank you Mathias, I will have a look at this.